Image-to-Video Generation via 3D Facial Dynamics

We present a versatile model, FaceAnime, for various video generation tasks from still images. Video single face image is an interesting problem and usually tackled by utilizing Generative Adversarial Networks (GANs) to integrate information the input sequence of sparse facial landmarks. However, generated images suffer quality loss, distortion, identity change, expression mismatching due weak representation capacity In this paper, we propose “imagine” according reconstructed 3D dynamics, aiming generate realistic identity-preserving video, with precisely predicted pose expression. The dynamics reveal changes motion, can serve as strong prior knowledge guiding highly generation. particular, explore prediction exploit well-designed dynamic network predict image. are then further rendered texture mapping algorithm recover structural details textures generating frames. Our model AR/VR entertainment applications, such retargeting prediction. Superior experimental results have well demonstrated its effectiveness in high-fidelity, identity-preserving, visually pleasant clips source